Joint use of static and dynamic software verification techniques: a cross-domain view in safety critical system industries

International audienceHow different are the approaches to combining formal methods (FM) and testing in the safety standards of the automotive, aeronautic, nuclear, process, railway and space industries? This is the question addressed in this paper by a cross-domain group of experts involved in the revision committees of ISO 26262, DO-178C, IEC 60880, IEC 61508, EN 50128 and ECSS-Q-ST-8OC. First we review some commonalities and differences regarding application of formal methods in theaforementioned standards. Are they mandatory or recommended only? What kind of properties are they advised to be applied to? What is specified in the different standards regarding coverage (both functional and structural) if testing and formal methods are used jointly?We also account for the return on experience of the group members in the six industrial domains regarding state of the art practice of joint use of formal methods and testing. Where did formal methods actually prove to outperform testing? Then we discuss verification coverage, and more specifically the role of structural coverage. Does structural coverage play the same role in all the standards? Is it specific to testing and irrelevant for formal methods? What verification terminationcriteria is applicable in case FM-test mix? We conclude on some prospective views on how software safety standards may evolve to maximize the benefits of joint use of dynamic (testing) and static (FM) verification methods

Joint use of static and dynamic software verification techniques: a cross-domain view in safety critical system industries

International audienceHow different are the approaches to combining formal methods (FM) and testing in the safety standards of the automotive, aeronautic, nuclear, process, railway and space industries? This is the question addressed in this paper by a cross-domain group of experts involved in the revision committees of ISO 26262, DO-178C, IEC 60880, IEC 61508, EN 50128 and ECSS-Q-ST-8OC. First we review some commonalities and differences regarding application of formal methods in theaforementioned standards. Are they mandatory or recommended only? What kind of properties are they advised to be applied to? What is specified in the different standards regarding coverage (both functional and structural) if testing and formal methods are used jointly?We also account for the return on experience of the group members in the six industrial domains regarding state of the art practice of joint use of formal methods and testing. Where did formal methods actually prove to outperform testing? Then we discuss verification coverage, and more specifically the role of structural coverage. Does structural coverage play the same role in all the standards? Is it specific to testing and irrelevant for formal methods? What verification terminationcriteria is applicable in case FM-test mix? We conclude on some prospective views on how software safety standards may evolve to maximize the benefits of joint use of dynamic (testing) and static (FM) verification methods